Field of the Invention
The present invention relates to an image processing apparatus, and particularly relates to an image processing technique for performing an optimum tone control for each object area within an image.
Description of the Related Art
Conventionally, a technique for locally controlling tones within an image has been known. In the case of performing local brightness compensation for a dark part, this technique is applied, for example, to perform compensation such that an object (especially, a person) that has been captured as a dark image in a backlit scene has a natural brightness. Such a technique is usually called dodging processing (tone compensation process) or the like.
Here, the dodging processing (tone compensation process) will be described in further detail. The dodging processing is achieved by locally obtaining, for luminance values within an input image, a gain from predetermined gain characteristics, and performing gain processing on the input image. The gain characteristics are set such that a larger gain is applied to a portion having a lower luminance. The reason for this is that an object to which the processing is to be applied is darkened in the above-described backlit scene, and thus, a large gain is applied to the low luminance portion including that object, thereby making the object bright in an output image.
However, the input image contains various objects. I the gain characteristics are directly used, for example, to an area with a very local luminance change, such as small shades of trees, the contrast to the surroundings is lost, resulting in an unnatural output image.
To address this problem, one method of dodging processing determines the gain by also using a more global brightness, instead of determining the gain simply by using a very local luminance value of the image. Specifically, the input image is divided into a plurality of images (hereinafter referred to as hierarchical images) having predetermined frequency bands. Then, as a global brightness, a low-frequency gain is calculated from a luminance value in a low-frequency hierarchical image (e.g., an image obtained by blurring the input image with a low-pass filter). Also, a high-frequency gain is calculated from a luminance value in a high-frequency hierarchical image (e.g., the luminance of the input image itself). Then, a final gain amount is calculated by weighting and adding the low-frequency gain and the high-frequency gain.
Examples of the weighting method include a method in which the gain is determined based on the size of the edge of the gain amount at a position of interest. The reason for using this method is that it is necessary to switch gains on a pixel-by-pixel basis so that the switching of the gains will not appear in an image, for example, at a boundary portion between an object darkened due to backlight and a bright sky. This problem becomes prominent when the gain significantly changes at the boundary portion, and therefore, the weight of using the high-frequency gain is increased with an increase in the size of the edge of the gain amount. In this manner, by adaptively changing the weight of using the low-frequency gain and the weight of using the high-frequency gain in each position within an image, a process for compensating for dark parts is performed, while suppressing the unnaturalness of the output image.
FIGS. 18A and 18B are diagrams showing an effect of dodging processing. In FIG. 18A, dodging processing is not performed, and therefore, a backlit person is darkened. On the other hand, in FIG. 18B, dodging processing is performed, and the backlit person is shown with natural brightness close to that it actually appears to be. FIGS. 19A and 19B are graphs showing example of the gain characteristics and the input/output characteristics of the dodging processing. FIG. 19B shows an example of the gain characteristics in which the gain is larger at a portion with a lower luminance. The gain characteristics are calculated from the input/output characteristics shown in FIG. 19A. In a backlit scene as described above, mainly a person area is present in a low luminance range in which the input luminance is Y0 or less, and the output image shows characteristics in which tone is assigned with priority to the low luminance range. The foregoing is the description of the dodging processing.
As the technique for further improving the quality of the output image in dodging processing, various techniques have been proposed. Japanese Patent Laid-Open No. 2014-153959 proposes a technique for adaptively calculating the gain characteristics from a distribution of luminance values of various objects contained in an image. In particular, determining the gain characteristics based on the relationship between the luminance values of the detected object areas is disclosed. Japanese Patent Laid-Open No. 2011-175608 discloses that different types of tone characteristics are set for each of the hierarchical images so as to intentionally saturate a local high luminance range.
However, with the conventional dodging processing, the tones of the other luminance ranges are compressed by an amount corresponding to the amount of compensation for the dark parts of the input image. Consequently, the tone characteristics of the compressed luminance ranges are lost. In the characteristics shown in FIGS. 19A and 19B, luminances belonging to a high luminance range of a luminance value Y1 or more are significantly compressed. In particular, the tone of the sky area and the like is compressed, resulting in a loss of contrast in the output image. The technique disclosed in Japanese Patent Laid-Open No. 2014-153959 analyzes the input image, and adaptively changes the luminance range that are to be compressed. However, some of the luminance ranges are eventually compressed, resulting in an output image in which the tone characteristics are partly lost.